How a Part of the Brain Might or Might Not Work: A New Hierarchical Model of Object Recognition

The classical model of visual processing in cortex is a hierarchy of increasingly sophisticated representations, extending in a natural way the model of simple to complex cells of Hubel and Wiesel. Somewhat surprisingly, little quantitative modeling has been done in the last 15 years to explore the biological feasibility of this class of models to explain higher level visual processing, such as object recognition in cluttered scenes. We describe a new hierarchical model, HMAX, that accounts well for this complex visual task, is consistent with several recent physiological experiments in inferotemporal cortex and makes testable predictions. Key to achieve invariance and robustness to clutter is a MAX-like response function of some model neurons which selects (an approximation to) the maximum activity over all the afferents, with interesting connections to “scanning” operations used in recent computer vision algorithms. We then turn to the question of object recognition in natural (“continuous”) object classes, such as faces, which recent physiological experiments have suggested are represented by a sparse distributed population code. We performed two psychophysical experiments in which subjects were trained to perform subordinate level discrimination in a continuous object class — images of computer-rendered cars — created using a 3D morphing system. By comparing the recognition performance of trained and untrained subjects we could estimate the effects of viewpoint-specific training and infer properties of the object class-specific representation learned as a result of training. We then compared the experimental findings to simulations in HMAX, to investigate the computational properties of a population-based object class representation. We find experimental evidence, supported by modeling results, that training builds a viewpointand class-specific representation that supplements a pre-existing representation with lower shape discriminability but greater viewpoint invariance. Finally, we show how HMAX can be extended in a straightforward fashion to perform object categorization and to support arbitrary class hierarchies. We demonstrate the capability of our scheme, called “Categorical Basis Functions” (CBF), with the example domain of cat/dog categorization, and apply it to study some recent findings in categorical perception. Thesis Supervisor: Tomaso Poggio Title: Uncas and Helen Whitaker Professor